Sociolinguistic factors affecting performance in the Clinical Skills Assessment of the MRCGP: a mixed-methods approach

Background: Differential performance in clinical skills assessments is a widespread phenomenon, for which there remain few explanations. Aim: To better understand the conversational contexts of simulated consultations and how candidates actually behave in these consultations and to determine sociolinguistic factors for highand low-performing candidates. Design & setting: Taking the Membership of the Royal College of General Practitioners’ (MRCGP) clinical skills assessment (CSA) examination as a model, this research applied sociolinguistic analyses to case videos of 198 consecutive candidates presenting for the CSA examination. Method: Using a mixed-methods approach, both quantitative and qualitative sociolinguistics methodologies were combined to analyse video consultations, and findings were compared with those from group discussions with MRCGP examiners. Results: There is more ‘talk’ in simulated consultations than in real life. On macroanalysis, there was little difference between poorand well-performing candidates. However, microanalysis found subtle differences in structuring consultations, metacommunication, picking up cues, and misunderstandings with and giving explanations to patients. Formulaic talk, contrary to examiners’ perceptions was more common in successful candidates, but it was personalised and sited appropriately in the consultation. Conclusion: This is an interactionally demanding form of clinical assessment, that requires giving support to candidates and a more analytic approach to the development of interpersonal skills. Sociolinguistic features of consulting to help trainers and candidates prepare for the CSA are identified. How this fits in Psychometric analysis of performance in the MRCGP’s CSA examination has failed to show the reasons for differential performance that has been the subject of much debate. It is part of a wider international picture in both undergraduate and postgraduate examinations. This sociolinguistic study applies a different lens to performance, allowing understanding and observation of detail of talk that would not otherwise be seen. It is of particular use to the training community and to candidates preparing to sit the CSA examination. Hawthorne K et al. BJGP Open 2017; DOI: 10.3399/bjgpopen17X100713 1 of 9 RESEARCH Introduction Differential performance in the CSA of the licensing MRCGP exams has been noted since the beginning of this assessment in 2007. UK black and minority ethnic graduates are less likely to pass than UK white graduates, and international medical graduates (IMGs) are less likely to pass than UK candidates. This research mirrors similar findings in many undergraduate and postgraduate examinations (both written articles and clinical tests) in medicine and other subjects in higher education, in the UK and other developed countries. Psychometric analyses of the marks have failed to show reasons for this difference. It does not appear to be due to bias on the part of examiners or role players (RPs). The researchers used a mixed-methods approach, combining both quantitative and qualitative data, and drawing on sociolinguistics methodology to look at the fine-grained detail of candidate performance. The aim of this research was to better understand the conversational contexts of simulated consultations, how the construct of the CSA determines candidate behaviour, and how candidates actually behave in the CSA. The terminology of ‘high’or ‘low’-performing candidates was used rather than assuming IMG candidates would perform less well than UK graduates. This study looks at some of the sociolinguistic factors that contrast highand low-performing candidates and outlines key areas where candidates need preparation for the CSA. It suggests learning strategies that will help both performance in the CSA and in day-to-day real-life consultations.